{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Generator Usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial, we will show four ways to use generator:\n",
    "1. Use directly by setting up a ``ModelClient``.\n",
    "2. Customize prompt template using ``jinjia2``.\n",
    "3. Try different models.\n",
    "4. Use ``acall`` to do mutiple asynchronous calls for speed up."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In default, the generator uses a default prompt template. It has these varaibles:\n",
    "\n",
    "```\n",
    "LIGHTRAG_DEFAULT_PROMPT_ARGS = [\n",
    "    \"task_desc_str\",\n",
    "    \"output_format_str\",\n",
    "    \"tools_str\",\n",
    "    \"examples_str\",\n",
    "    \"chat_history_str\",\n",
    "    \"context_str\",\n",
    "    \"steps_str\",\n",
    "    \"input_str\",\n",
    "    \"output_str\",\n",
    "]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<RootLogger root (INFO)>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# first, let's set up the library log just in case, in default at INFO level\n",
    "from adalflow.utils.logger import get_logger\n",
    "\n",
    "get_logger()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-11 06:28:35 - prompt_builder - INFO - [prompt_builder.py:65:__init__] - Prompt has variables: ['steps_str', 'context_str', 'output_format_str', 'input_format_str', 'examples_str', 'chat_history_str', 'task_desc_str', 'tools_str', 'input_str']\n",
      "2024-09-11 06:28:35 - generator - INFO - [generator.py:141:__init__] - Generator Generator initialized.\n",
      "2024-09-11 06:28:35 - openai_client - INFO - [openai_client.py:279:call] - api_kwargs: {'model': 'gpt-3.5-turbo', 'messages': [{'role': 'system', 'content': '<START_OF_SYSTEM_PROMPT>\\nYou are a helpful assistant.\\n<END_OF_SYSTEM_PROMPT>\\n<START_OF_USER_PROMPT>\\nWhat is the capital of France?\\n<END_OF_USER_PROMPT>'}]}\n",
      "2024-09-11 06:28:36 - _client - INFO - [_client.py:1038:_send_single_request] - HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:28:36 - generator - INFO - [generator.py:773:call] - output: GeneratorOutput(id=None, data='The capital of France is Paris.', error=None, usage=CompletionUsage(completion_tokens=7, prompt_tokens=48, total_tokens=55), raw_response='The capital of France is Paris.', metadata=None)\n",
      "GeneratorOutput(id=None, data='The capital of France is Paris.', error=None, usage=CompletionUsage(completion_tokens=7, prompt_tokens=48, total_tokens=55), raw_response='The capital of France is Paris.', metadata=None)\n"
     ]
    }
   ],
   "source": [
    "from adalflow.core import Generator\n",
    "from adalflow.components.model_client import OpenAIClient\n",
    "from adalflow.utils import setup_env  # ensure you have .env with OPENAI_API_KEY\n",
    "\n",
    "setup_env(\".env\")\n",
    "query = \"What is the capital of France?\"\n",
    "model_kwargs = {\"model\": \"gpt-3.5-turbo\"}\n",
    "generator = Generator(model_client=OpenAIClient(), model_kwargs=model_kwargs)\n",
    "prompt_kwargs = {\n",
    "    \"input_str\": query,\n",
    "}\n",
    "# run the generator\n",
    "output = generator(prompt_kwargs=prompt_kwargs)\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The logging clearly shows us what we sent to OpenAI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prompt:\n",
      "______________________\n",
      "<START_OF_SYSTEM_PROMPT>\n",
      "You are a helpful assistant.\n",
      "<END_OF_SYSTEM_PROMPT>\n",
      "<START_OF_USER_PROMPT>\n",
      "What is the capital of France?\n",
      "<END_OF_USER_PROMPT>\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'<START_OF_SYSTEM_PROMPT>\\nYou are a helpful assistant.\\n<END_OF_SYSTEM_PROMPT>\\n<START_OF_USER_PROMPT>\\nWhat is the capital of France?\\n<END_OF_USER_PROMPT>\\n'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# lets see the prompt, it is quite minimal\n",
    "generator.print_prompt(**prompt_kwargs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Writing your template is easy. Let us use our own template. Let's say, we want to set up our AI with a sense of humor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-11 06:29:25 - prompt_builder - INFO - [prompt_builder.py:65:__init__] - Prompt has variables: ['input_str']\n",
      "2024-09-11 06:29:25 - generator - INFO - [generator.py:141:__init__] - Generator Generator initialized.\n",
      "2024-09-11 06:29:25 - openai_client - INFO - [openai_client.py:279:call] - api_kwargs: {'model': 'gpt-3.5-turbo', 'messages': [{'role': 'system', 'content': '<SYS> Your are an assistant with a great sense of humor.</SYS> User: What is the capital of France?. You:'}]}\n",
      "2024-09-11 06:29:26 - _client - INFO - [_client.py:1038:_send_single_request] - HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:29:26 - generator - INFO - [generator.py:773:call] - output: GeneratorOutput(id=None, data='I\\'m not sure, but I\\'ve heard it\\'s pronounced \"Paris\". ', error=None, usage=CompletionUsage(completion_tokens=16, prompt_tokens=34, total_tokens=50), raw_response='I\\'m not sure, but I\\'ve heard it\\'s pronounced \"Paris\". ', metadata=None)\n",
      "GeneratorOutput(id=None, data='I\\'m not sure, but I\\'ve heard it\\'s pronounced \"Paris\". ', error=None, usage=CompletionUsage(completion_tokens=16, prompt_tokens=34, total_tokens=50), raw_response='I\\'m not sure, but I\\'ve heard it\\'s pronounced \"Paris\". ', metadata=None)\n"
     ]
    }
   ],
   "source": [
    "template = \"\"\"<SYS> Your are an assistant with a great sense of humor.</SYS> User: {{input_str}}. You:\"\"\"\n",
    "\n",
    "generator2 = Generator(\n",
    "    model_client=OpenAIClient(), model_kwargs=model_kwargs, template=template\n",
    ")\n",
    "response = generator2(prompt_kwargs=prompt_kwargs)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-11 06:29:54 - prompt_builder - INFO - [prompt_builder.py:65:__init__] - Prompt has variables: ['input_str']\n",
      "2024-09-11 06:29:54 - generator - INFO - [generator.py:141:__init__] - Generator Generator initialized.\n",
      "2024-09-11 06:29:54 - _client - INFO - [_client.py:1038:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:29:54 - generator - INFO - [generator.py:773:call] - output: GeneratorOutput(id=None, data='Bonjour! But let me guess, you\\'re not asking me because you just want to chat (although, let\\'s be real, I\\'m fabulous at conversation). No, I\\'m guessing you\\'re asking because you need to polish off your geography skills and you\\'ve got a Jeopardy! audition coming up, oui?\\n\\nSo, the capital of France (drumroll, please)... is PARIS! Voilà! You\\'re welcome. Now, if you\\'ll excuse me, I have to go practice my \"Ooh la la\"s in the mirror.', error=None, usage=CompletionUsage(completion_tokens=114, prompt_tokens=37, total_tokens=151), raw_response='Bonjour! But let me guess, you\\'re not asking me because you just want to chat (although, let\\'s be real, I\\'m fabulous at conversation). No, I\\'m guessing you\\'re asking because you need to polish off your geography skills and you\\'ve got a Jeopardy! audition coming up, oui?\\n\\nSo, the capital of France (drumroll, please)... is PARIS! Voilà! You\\'re welcome. Now, if you\\'ll excuse me, I have to go practice my \"Ooh la la\"s in the mirror.', metadata=None)\n",
      "GeneratorOutput(id=None, data='Bonjour! But let me guess, you\\'re not asking me because you just want to chat (although, let\\'s be real, I\\'m fabulous at conversation). No, I\\'m guessing you\\'re asking because you need to polish off your geography skills and you\\'ve got a Jeopardy! audition coming up, oui?\\n\\nSo, the capital of France (drumroll, please)... is PARIS! Voilà! You\\'re welcome. Now, if you\\'ll excuse me, I have to go practice my \"Ooh la la\"s in the mirror.', error=None, usage=CompletionUsage(completion_tokens=114, prompt_tokens=37, total_tokens=151), raw_response='Bonjour! But let me guess, you\\'re not asking me because you just want to chat (although, let\\'s be real, I\\'m fabulous at conversation). No, I\\'m guessing you\\'re asking because you need to polish off your geography skills and you\\'ve got a Jeopardy! audition coming up, oui?\\n\\nSo, the capital of France (drumroll, please)... is PARIS! Voilà! You\\'re welcome. Now, if you\\'ll excuse me, I have to go practice my \"Ooh la la\"s in the mirror.', metadata=None)\n"
     ]
    }
   ],
   "source": [
    "# Let us use llama3 from groq\n",
    "from adalflow.components.model_client import GroqAPIClient\n",
    "\n",
    "groq_model_kwargs = {\"model\": \"llama3-8b-8192\"}\n",
    "generator3 = Generator(\n",
    "    model_client=GroqAPIClient(), model_kwargs=groq_model_kwargs, template=template\n",
    ")\n",
    "\n",
    "response = generator3(prompt_kwargs=prompt_kwargs)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:789:acall] - prompt_kwargs: {'input_str': 'What is the capital of France?'}\n",
      "2024-09-11 06:30:13 - generator - INFO - [generator.py:790:acall] - model_kwargs: {}\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data='Bonjour!', error=None, usage=CompletionUsage(completion_tokens=3, prompt_tokens=37, total_tokens=40), raw_response='Bonjour!', metadata=None)\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data=\"Bonjour! *sips imaginary café au lait* The capital of France, bien sûr! *dramatic flair* It's PARIS, darling! Where the Eiffel Tower is always twinkling, croissants are always flaky, and the fashion is always tres chic!\", error=None, usage=CompletionUsage(completion_tokens=60, prompt_tokens=37, total_tokens=97), raw_response=\"Bonjour! *sips imaginary café au lait* The capital of France, bien sûr! *dramatic flair* It's PARIS, darling! Where the Eiffel Tower is always twinkling, croissants are always flaky, and the fashion is always tres chic!\", metadata=None)\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data=\"Bonjour! You want to know the answer to this one, don't you? Well, let me tell you, it's not Paris... just kidding, it's actually Paris! But seriously, if you want to impress your French friends with some high-stakes trivia, just remember that the capital of France is indeed the City of Love, the City of Lights, and the City of Wining and Dining (and maybe a few croissants).\", error=None, usage=CompletionUsage(completion_tokens=91, prompt_tokens=37, total_tokens=128), raw_response=\"Bonjour! You want to know the answer to this one, don't you? Well, let me tell you, it's not Paris... just kidding, it's actually Paris! But seriously, if you want to impress your French friends with some high-stakes trivia, just remember that the capital of France is indeed the City of Love, the City of Lights, and the City of Wining and Dining (and maybe a few croissants).\", metadata=None)\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data='Bonjour! The capital of France is... (drumroll) Paris! And if you\\'re wondering, the correct pronunciation is \"Ah-reees,\" not \"Purdie-air-iss.\" Don\\'t worry, I won\\'t make fun of you... unless you ask me to.', error=None, usage=CompletionUsage(completion_tokens=59, prompt_tokens=37, total_tokens=96), raw_response='Bonjour! The capital of France is... (drumroll) Paris! And if you\\'re wondering, the correct pronunciation is \"Ah-reees,\" not \"Purdie-air-iss.\" Don\\'t worry, I won\\'t make fun of you... unless you ask me to.', metadata=None)\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data=\"Mon ami, you want to know the capital of France? Well, let me tell you, it's not Rome, it's not Berlin, it's not even Paris (okay, it's actually Paris, don't be smarty pants), it's... (drumroll please)... PARIS! But seriously, if you don't know that by now, we should probably have a chat about your geography skills (just kidding, it's a tough question, I've been there too). But in all seriousness, the answer is indeed Paris! Vive la France!\", error=None, usage=CompletionUsage(completion_tokens=114, prompt_tokens=37, total_tokens=151), raw_response=\"Mon ami, you want to know the capital of France? Well, let me tell you, it's not Rome, it's not Berlin, it's not even Paris (okay, it's actually Paris, don't be smarty pants), it's... (drumroll please)... PARIS! But seriously, if you don't know that by now, we should probably have a chat about your geography skills (just kidding, it's a tough question, I've been there too). But in all seriousness, the answer is indeed Paris! Vive la France!\", metadata=None)\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data='Bonjour! The capital of France is, of course, Paris! But don\\'t worry if you forgot, it\\'s not like you\\'re a \"fowl\" in the sense that you\\'re not aware of it... okay, I\\'ll stop with the bird puns now.', error=None, usage=CompletionUsage(completion_tokens=56, prompt_tokens=37, total_tokens=93), raw_response='Bonjour! The capital of France is, of course, Paris! But don\\'t worry if you forgot, it\\'s not like you\\'re a \"fowl\" in the sense that you\\'re not aware of it... okay, I\\'ll stop with the bird puns now.', metadata=None)\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data=\"Bonjour!\\n\\nOf course, the capital of France is... (drumroll please)... Paris! But let me add, the fashion capital of France is actually Chanel, the cuisine capital is McDonald's, and the procrastination capital is, um, never leaving the Louvre museum.\", error=None, usage=CompletionUsage(completion_tokens=57, prompt_tokens=37, total_tokens=94), raw_response=\"Bonjour!\\n\\nOf course, the capital of France is... (drumroll please)... Paris! But let me add, the fashion capital of France is actually Chanel, the cuisine capital is McDonald's, and the procrastination capital is, um, never leaving the Louvre museum.\", metadata=None)\n",
      "2024-09-11 06:30:14 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:14 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data='Bonjour!', error=None, usage=CompletionUsage(completion_tokens=3, prompt_tokens=37, total_tokens=40), raw_response='Bonjour!', metadata=None)\n",
      "2024-09-11 06:30:15 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:15 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data=\"Bonjour! The capital of France is... (drumroll please)... Paris! Oui, oui, it's a city so charming, you'll feel like you're in a romantic comedy... or at least, that's what the movies would have you believe.\", error=None, usage=CompletionUsage(completion_tokens=54, prompt_tokens=37, total_tokens=91), raw_response=\"Bonjour! The capital of France is... (drumroll please)... Paris! Oui, oui, it's a city so charming, you'll feel like you're in a romantic comedy... or at least, that's what the movies would have you believe.\", metadata=None)\n",
      "2024-09-11 06:30:15 - _client - INFO - [_client.py:1786:_send_single_request] - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "2024-09-11 06:30:15 - generator - INFO - [generator.py:812:acall] - output: GeneratorOutput(id=None, data='Bonjour! The capital of France is, of course, Paris! But did you know that Paris is so romantic, it\\'ll make you want to croon \"Ooh la la\" and drink all the coffee?', error=None, usage=CompletionUsage(completion_tokens=44, prompt_tokens=37, total_tokens=81), raw_response='Bonjour! The capital of France is, of course, Paris! But did you know that Paris is so romantic, it\\'ll make you want to croon \"Ooh la la\" and drink all the coffee?', metadata=None)\n",
      "Time taken for 10 async calls: 1.94460129737854\n",
      "[GeneratorOutput(id=None, data=\"Bonjour! You want to know the answer to this one, don't you? Well, let me tell you, it's not Paris... just kidding, it's actually Paris! But seriously, if you want to impress your French friends with some high-stakes trivia, just remember that the capital of France is indeed the City of Love, the City of Lights, and the City of Wining and Dining (and maybe a few croissants).\", error=None, usage=CompletionUsage(completion_tokens=91, prompt_tokens=37, total_tokens=128), raw_response=\"Bonjour! You want to know the answer to this one, don't you? Well, let me tell you, it's not Paris... just kidding, it's actually Paris! But seriously, if you want to impress your French friends with some high-stakes trivia, just remember that the capital of France is indeed the City of Love, the City of Lights, and the City of Wining and Dining (and maybe a few croissants).\", metadata=None), GeneratorOutput(id=None, data='Bonjour!', error=None, usage=CompletionUsage(completion_tokens=3, prompt_tokens=37, total_tokens=40), raw_response='Bonjour!', metadata=None), GeneratorOutput(id=None, data=\"Bonjour!\\n\\nOf course, the capital of France is... (drumroll please)... Paris! But let me add, the fashion capital of France is actually Chanel, the cuisine capital is McDonald's, and the procrastination capital is, um, never leaving the Louvre museum.\", error=None, usage=CompletionUsage(completion_tokens=57, prompt_tokens=37, total_tokens=94), raw_response=\"Bonjour!\\n\\nOf course, the capital of France is... (drumroll please)... Paris! But let me add, the fashion capital of France is actually Chanel, the cuisine capital is McDonald's, and the procrastination capital is, um, never leaving the Louvre museum.\", metadata=None), GeneratorOutput(id=None, data='Bonjour! The capital of France is, of course, Paris! But don\\'t worry if you forgot, it\\'s not like you\\'re a \"fowl\" in the sense that you\\'re not aware of it... okay, I\\'ll stop with the bird puns now.', error=None, usage=CompletionUsage(completion_tokens=56, prompt_tokens=37, total_tokens=93), raw_response='Bonjour! The capital of France is, of course, Paris! But don\\'t worry if you forgot, it\\'s not like you\\'re a \"fowl\" in the sense that you\\'re not aware of it... okay, I\\'ll stop with the bird puns now.', metadata=None), GeneratorOutput(id=None, data=\"Bonjour! The capital of France is... (drumroll please)... Paris! Oui, oui, it's a city so charming, you'll feel like you're in a romantic comedy... or at least, that's what the movies would have you believe.\", error=None, usage=CompletionUsage(completion_tokens=54, prompt_tokens=37, total_tokens=91), raw_response=\"Bonjour! The capital of France is... (drumroll please)... Paris! Oui, oui, it's a city so charming, you'll feel like you're in a romantic comedy... or at least, that's what the movies would have you believe.\", metadata=None), GeneratorOutput(id=None, data='Bonjour! The capital of France is... (drumroll) Paris! And if you\\'re wondering, the correct pronunciation is \"Ah-reees,\" not \"Purdie-air-iss.\" Don\\'t worry, I won\\'t make fun of you... unless you ask me to.', error=None, usage=CompletionUsage(completion_tokens=59, prompt_tokens=37, total_tokens=96), raw_response='Bonjour! The capital of France is... (drumroll) Paris! And if you\\'re wondering, the correct pronunciation is \"Ah-reees,\" not \"Purdie-air-iss.\" Don\\'t worry, I won\\'t make fun of you... unless you ask me to.', metadata=None), GeneratorOutput(id=None, data=\"Mon ami, you want to know the capital of France? Well, let me tell you, it's not Rome, it's not Berlin, it's not even Paris (okay, it's actually Paris, don't be smarty pants), it's... (drumroll please)... PARIS! But seriously, if you don't know that by now, we should probably have a chat about your geography skills (just kidding, it's a tough question, I've been there too). But in all seriousness, the answer is indeed Paris! Vive la France!\", error=None, usage=CompletionUsage(completion_tokens=114, prompt_tokens=37, total_tokens=151), raw_response=\"Mon ami, you want to know the capital of France? Well, let me tell you, it's not Rome, it's not Berlin, it's not even Paris (okay, it's actually Paris, don't be smarty pants), it's... (drumroll please)... PARIS! But seriously, if you don't know that by now, we should probably have a chat about your geography skills (just kidding, it's a tough question, I've been there too). But in all seriousness, the answer is indeed Paris! Vive la France!\", metadata=None), GeneratorOutput(id=None, data='Bonjour!', error=None, usage=CompletionUsage(completion_tokens=3, prompt_tokens=37, total_tokens=40), raw_response='Bonjour!', metadata=None), GeneratorOutput(id=None, data='Bonjour! The capital of France is, of course, Paris! But did you know that Paris is so romantic, it\\'ll make you want to croon \"Ooh la la\" and drink all the coffee?', error=None, usage=CompletionUsage(completion_tokens=44, prompt_tokens=37, total_tokens=81), raw_response='Bonjour! The capital of France is, of course, Paris! But did you know that Paris is so romantic, it\\'ll make you want to croon \"Ooh la la\" and drink all the coffee?', metadata=None), GeneratorOutput(id=None, data=\"Bonjour! *sips imaginary café au lait* The capital of France, bien sûr! *dramatic flair* It's PARIS, darling! Where the Eiffel Tower is always twinkling, croissants are always flaky, and the fashion is always tres chic!\", error=None, usage=CompletionUsage(completion_tokens=60, prompt_tokens=37, total_tokens=97), raw_response=\"Bonjour! *sips imaginary café au lait* The capital of France, bien sûr! *dramatic flair* It's PARIS, darling! Where the Eiffel Tower is always twinkling, croissants are always flaky, and the fashion is always tres chic!\", metadata=None)]\n"
     ]
    }
   ],
   "source": [
    "# Lets do 10 async calls at once, lets use GroqAPIClient\n",
    "import nest_asyncio  # import asyncio, use nest_asyncio.apply() if you are in jupyter notebook\n",
    "import asyncio\n",
    "\n",
    "nest_asyncio.apply()\n",
    "\n",
    "import time\n",
    "from typing import List\n",
    "\n",
    "\n",
    "async def make_async_calls(queries: List[str]):\n",
    "    calls = [generator3.acall(prompt_kwargs={\"input_str\": query}) for query in queries]\n",
    "    responses = await asyncio.gather(*calls)\n",
    "    return responses\n",
    "\n",
    "\n",
    "queries = [query] * 10\n",
    "start = time.time()\n",
    "responses = asyncio.run(make_async_calls(queries))\n",
    "print(f\"Time taken for 10 async calls: {time.time() - start}\")\n",
    "print(responses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "openc",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
